Because a 24-Hz video such as a cinema video cannot be displayed as-is by a display device displaying a 60 Hz video, conventional technologies perform a frame rate conversion process where a video such as a movie film having images recorded at 24 frames per second is converted into a video signal containing 30 frames per second (60 fields) which is used in TV broadcasting.
Examples of such frame rate conversion process include an image processing apparatus disclosed in Patent Document 1, which detects a motion vector by frames n−1, n with the narrowest inter-frame distance among combinations of 2-frames sandwiching an interpolation frame, to associate the motion vector with pixels of the interpolation frame. In the case where there remain pixels with which no motion vector can be associated, two frames, at least either one of which differs from the 2-frames above (e.g., frames n−2, n+1 whose inter-frame distance is wider than that of the frames n−1, n), are used to detect the motion vector, which is associated with the pixels of the interpolation frame. In the case where there still remain pixels with which no motion vector can be associated, two-frames, at least either one of which differs from the 2-frames above (e.g., frames n−3, n+2 whose inter-frame distance is wider than that of the frames n−2, n+1), are used to detect the motion vector, which is associated with the pixels of the interpolation frame. This image processing apparatus can prevent deterioration of the image quality of the interpolation frame by associating motion vectors with as many pixels of an interpolation frame as possible.
Another example of the frame rate conversion process is a video signal system conversion device disclosed in Patent Document 2. A conventional video processing apparatus typified by Patent Document 2 performs frame rate conversion by using a motion vector detection unit and motion vector processing unit. Next is described a process where the conventional video processing apparatus converts a so-called cinema video having a 24-Hz input frame frequency, into a video having a 60-Hz output frame frequency.
FIG. 8 is a block diagram showing a configuration of a conventional video processing apparatus that performs frame rate conversion. FIG. 9 is a block diagram showing a configuration of a motion vector processing unit shown in FIG. 8.
In the conventional video processing apparatus shown in FIG. 8, a frame rate conversion process is executed using a motion vector detection unit 101 and motion vector processing unit 102. More specifically, the motion vector detection unit 101 uses image data of a continuous input frame n and image data of an input frame n+1, to detect a motion amount between these frames, and detect a motion vector V on a pixel to pixel basis or a block to block basis. The motion vector processing unit 102 generates image data of an intermediate frame n+K (where K represents an interpolation phase coefficient, and 0≦K<1) by using the motion vector, the image data of the input frame n and the image data of the input frame n+1.
As shown in FIG. 9, the motion vector processing unit 102 has an interpolation phase calculation unit 103, multipliers 104, 105, subtracter 106, projection processing units 107, 108, and merge unit 109. First, the interpolation phase calculation unit 103 adds the result of a calculation, input frame frequency/output frame frequency (=24/60=0.4), for each processes, and obtains the fractional portion thereof as the interpolation phase coefficient K.
Here, the value 0.4 is added sequentially to an initial value 0.0 and so on to obtain the results 0.0, 0.4, 0.8, 1.2, 1.6, 2.0, . . . and the like. The interpolation phase coefficients K obtained as the fractional portions of these results are 0.0, 0.4, 0.8, 0.2, 0.6, 0.0, and these five patterns are repeated. In addition, control is performed such that the input frame is switched sequentially when the integer portions of the added results above are changed.
The motion vector V detected by the motion vector detection unit 101 applies when the inter-frame distance is 1.0. Therefore, when generating the image data of the intermediate frame n+K by performing a projection process based on the image data of the input frame n, the motion vector V is multiplied by the interpolation phase coefficient K, to perform a gain process. Similarly, when generating the image data of the intermediate frame n+K by performing the projection process based on the image data of the input frame n+1, the motion vector V is multiplied by −(1−K)=K−1, to perform the gain process. The symbols change in this case because the images are projected in a direction of going against the time from the input frame n+1.
As described above, the image data of the input frame n and the image data of the input frame n+1 are subjected to the projection process by the projection processing units 107, 108, to generate intermediate frame vides of these image data. These intermediate frame videos are merged accordingly by the merge unit 109, and the image data of the intermediate frame n+K are eventually output.
More specifically, in case of converting a frame rate from 24 Hz to 60 Hz, when sample phases on a time axis of a frame image to be input are 0.0, 1.0, 2.0, 3.0 and the like, interpolation phases on a time axis of a frame image to be output are 0.0, 0.4, 0.8, 1.2, 1.6, 2.0 and the like. A phase interval of these interpolation phases is 0.4 and determined by the calculation, input frame frequency/output frame frequency, which is 24/60 in this example. The motion vector processing unit 102 multiplies the motion vector V by each of the interpolation phase coefficients K, which are the fractional portions (0.0, 0.4, 0.8, 0.2, 0.6, . . . ) of the interpolation phases, projects the image data of the input frames onto the interpolation phases, to generate the image data of each intermediate frame.
The frame rate conversion process described above, so-called cinema smoothing process (a cinema video dejudder process), has the effect of converting a jumpy movement (judder) of a 24-Hz video (cinema video) into a smooth movement and is installed in the recent TV devices. This conventional cinema smoothing process is effective in relatively slow videos or videos in which the entire screens moves in the same direction.
However, for a video having a plurality of objects moving relatively fast in different directions on the screen, the conventional cinema smoothing process often damages a generated intermediate frame video due to various reasons. This video damage occurs as a result of an error in detecting a motion vector in the vicinity of a border between the objects, an error in detecting a motion vector of an object moving at high speed, and other errors in detecting motion vectors. Resolving these errors in detecting motion vectors is extremely difficult and consumes resources to resolve these problems.    Patent Document 1: Japanese patent application Publication No. 2007-288681    Patent Document 2: Japanese patent Publication No. 4083265